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Entropy is a concept that has traditionally been reliant on a definite notion of causality. However, 
without a definite notion of causality, the concept of entropy is not all lost. Indefinite causal 
structure results from combining probabilistic predictions and dynamical space-time. Combining the 
probabilistic nature of quantum theory and dynamical treatment space-time from general relativity 
is an approach to the problem of quantum gravity. The causaloid framework lays the mathematical 
groundwork to be able to treat indefinite causal structure. In this paper, we build on the causaloid 
mathematics and define a causally-unbiased entropy for an indefinite causal structure. In defining a 
causally-unbiased entropy, there comes about an emergent idea of causality in the form of a measure 
of causal connectedness, termed the Q factor. 



I. INTRODUCTION 

In Newtonian physics, physical processes are understood with respect to a fixed spatial coordinate system and a time 
parameter, which is absolute and ever increasing. Predictions are entirely deterministic. Quantum theory and general 
relativity depart from this classical picture in opposing manners. Quantum theory gives probabilistic predictions 
as to the outcomes of measurements, but retains fixed space and time coordinates. On the other hand, general 
relativity is deterministic, but shows that space and time form a dynamical structure. Reconciling these fundamental 
philosophical differences is one of the many challenges one is faced with in trying to construct a theory of quantum 
gravity. There have been many different approaches to this problem with many different results [TJ [2| U IH |5j [6]. 
One way of moving forward is to dismiss classical assumptions and create a probabilistic theory that has a dynamic 
causal structure. However, what results is indefinite casual structure. This is more radical than either probabilistic 
predictions or dynamical space-time structure. In general relativity, a separation between space-time locations is 
cither space- like or time-like. An indefinite causal structure would allow for a separation between space-time locations 
to be something like a quantum superposition of a space-like and a time-like separation. While we may be uncertain 
of the causal structure of the path between measurements, we know where in space-time we make measurements, what 
measurements we have made, and what outcomes we get. With this data, we can examine probabilistic correlations 
for information. The causaloid framework ([7], [8], [9]) provides us with the necessary structure. We will outline the 
essentials of this framework in Section 2. 

It is natural in discussions of causal structure to raise the question of entropy. The second law of thermodynamics 
tells us that in an isolated system, entropy can increase or remain the same, but it can never decrease |10j . In 
information theory, entropy is viewed as being a measure of uncertainty before we measure a state or equivalcntly, 
the amount of information gained by upon learning the state of a system . Inherent in both concepts of entropy 
is an assumed causal structure, specifically that there exists a background time. The standard definition of entropy 
is in the context of a definite causal structure with reference to absolute time. In order to make sense of entropy in 
an indefinite causal structure, a clear definition must be established. To do so requires consideration of the following 
questions: 

What are the concepts from the usual picture of entropy in a definite causal structure that are necessary 
to define entropy? What are the analogues to these concepts in a picture with indefinite causal structure? 

Using the formalism introduced in the causaloid framework, we are able to provide answers to these questions and 
then, define a causally-unbiased entropy. 
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In Section 2, we will review the relevant aspects of the causaloid framework. We then proceed with the new 
developments. In Section 3, we define a new type of product that is utilized in the work on entropy. The definition 
of causally-unbiased entropy and resulting features are developed in Section 4. 

II. CAUSALOID FRAMEWORK 
A. The Picture 

Every experiment results in a set of data from making measurements on a system. Each piece of data could be 
thought of as a card with three pieces of information on it; where the measurement is made in space-time, what is 
measured, and what the result of the measurement is. We will represent each card (or piece of data) as (x,f x ,y x ) 
where x denotes the space-time information, f x denotes the information pertinent to a choice of measurement or 
action, and y x denotes the information regarding an observation or outcome of a measurement. The set of all possible 
cards (i.e. all possible measurements with all possible outcomes with every space-time configuration) is denoted V. 
We can imagine running an experiment an infinite number of times so as to be able to obtain relative frequencies. In 
order for the cards to tell us the relative frequencies, we must systematically sort them. 

Each distinct x is defined as an elementary region of space-time. A composite region, denoted Ox, is a set of 
elementary regions. (Note: These definitions of "elementary region" and "composite region" differ from those in 
[II EE].) Therefore, these cards can be sorted according to their associated space-time region. The set of all possible 
cards with the same space-time information x written on them is the measurement information for elementary region 
x. We denote this set as R x . The measurement information for composite region 0\ is the union of all sets of 
measurement information for the elementary regions contained within the composite region. More concisely, 

Rx= (J R x (1) 

xeo x 

We can further sort the measurement information in a region. The procedure in a region, denoted F x , is the set of all 
distinct choices of measurement recorded for the region x. 

F x = [J (x,f x ,y x ) (2) 

all fx, V a: 

For composite regions, we define the following set: 

Fi= |J F x (3) 

Similarly, the outcome set in a region, denoted Y x is defined to be the set of all distinct outcomes of a measurement 
recorded for the region x. 

Y x = (J (x,f x ,y x ) (4) 

ally-c 

Again, for composite regions, we define 

Yi= \JY X (5) 

xed 

Notice that Y x C F x C R x and Y\ C F\ C R^. The composite structure we expected of our space-time regions is 
reflected structure throughout these sets. A set of cards with the measurement information for a region has no more 
or less structure than an elementary region of space-time. Therefore, without adding structure or losing generality, 
we can take the sets R x to be elementary regions, at least, for the purposes of this paper. From this point forward, 
the term region will be used interchangeably to refer to objects of type x or 0\ and type R x or R\. 
Notice that the set of all cards V can be viewed as all the cards from all (elementary) regions. 

V = |J R x (6) 

all x 

So V is the largest of all regions that can be considered. 

These definitions provide a firm foundation on which the causaloid framework rests both mathematically and 
conceptually. 
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B. First level physical compression 



The most basic quantity that we would want to be able to calculate is the probability that a certain (set of) 
outcome(s) is observed given that a certain (set of) measurement (s) has been performed at a (set of) location(s) in 
space and time. Suppose that the set of locations we are interested in is 0\. The set of all the cards corresponding 
to these locations called R\. We write pairings of measurements and corresponding outcomes in R\ as (Yi,i*i). A 
specific outcome and measurement pair is denoted as a\ (or equivalently, (Y" 1 , F" 1 )). The set of all a± in region 0\ 
is Tj. The set comprised of all the cards not in R\ is V — R±. We call (Yv— Rl , Fv-Ri) the generalized preparation 
because it is the information that surrounds R\ not only from the immediate past, but from the future and the rest 
of space-time as well. By the choices we make in setting up the experiment, we can put conditions on the generalized 
preparation such that Prob(lVI-FV) is well-defined. (See Ref.|7] for details.) Then we can write 



Prob(y v |iV) = Prob(y l5 Y v - Rl (i^iV^) 
For a specific pair ot\ (Y™ 1 , F" 1 ), we can write this probability as 

¥rob{Y?\Y v _ Rl \F?\F v _ Rl ) 



(7) 



(8) 



We use the short-hand p ai to denote the probability defined in Eq.(|8|. One way to specify the state of a system is to 
list all the possible p ai for elements of R x . 



( ■ \ 



ol\ e Ti 



(9) 



However, this over-specifies the state. We do not usually need to know the probability of every outcome of every 
measurement in order to determine what the complete state of the system is. Physical theories tell us what relation- 
ships exist between variables and what constraints those relationships place on the variables of the system. These 
relationships and constraints can be used to determine a reduced set of probabilities from which all other probabilities 
can be represented. The reduced set of probabilities is defined such that any probability can be written as a linear 
combination of the probabilities in the reduced set. Let us denote the reduced or fiducial set in R\ as f^i C Ti. This 
process of going from the set of all the probabilities to the smallest essential set we call first level physical compression. 
This can be expressed as 



such that 



( ] ) 

Ph 

V '■ J 



p al 



(10) 



(11) 



where r Ql encodes the physical compression and therefore, is determined by the details of the physical theory. We 
can define a decompression matrix, such that 



A' 1 = r I 



(12) 



where r Ql | ; means the l\ component of r Ql . 



C. Second level physical compression and the causaloid product 

Let us consider two distinct regions i?2 C V . In a similar fashion to the single region case, 

Paia2 = ¥*oh{Y?\Y^,Y v _ R ,_ R2 \F?\F2\F v _ Rl _ R2 ) 



(13) 
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(a) (b) 

FIG. 1: (a) A single region R\ in V (b) Two regions R\ and R 2 in V 



We specify the state of the system by listing all p ai a 2 - 

( ) 

Paia.2 

V J 

where x is the cartesian product. It can be shown that 



aia 2 € Ti x T 2 



(14) 



(15) 



which implies that the following list of probabilities is sufficient. 



Phh 

V J 



l\l 2 S ft i X ft 2 C Ti X T 2 



(16) 



This is effectively first level compression on each index. But if a physical theory has some connection between the 
two regions, f^i x f2 2 may no longer be the smallest set that is sufficient to represent all possible states. Then second 
level physical compression is possible. It is defined to be 



( ) 

Pkik 2 

V : / 



kik 2 £ 12 C fix x n 2 



such that 



Pa±a 2 



>l«2 ' P — ^ , r a 1 a 2 \k 1 k 2 Pkik 2 

K\ k 2 £E £^12 



(17) 



(18) 



When Q,i2 = ^i x f2 2 , second level compression is trivial. But it is proven in [8 j that it is possible that f2i 2 C £l\ x £7 2 . 
Now we can define a second level decompression matrix. By comparing Eq.(15) and Eq.(18), we infer that 



La l«2 \ klk2 



where 



\k x k 2 I 

yi ii/ 2 _rz i'2|fe 1 fe 2 



(19) 



(20) 



which is the desired second level decompression matrix. This matrix encodes how we move from pz 1 ; 2 's to Pk^'s. 
Using the definition of the first level decompression matrix, Eq.(19l becomes 



«1»2 | felfc2 



EA felfe2 r I r I 



(21) 



!ii 2 efiixfi2 
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This defines the causaloid product, denoted r Ql ® r a . 2 which unifies the different causal structure-specific products. 
Explicitly, 

^ot\ ® ^ Oi2 — ^"aia2 (22) 

It is this product that allows us to look at the probabilistic correlations between arbitrary locations in space-time 
without specifying the causal relationship. 

We have shown second level compression for the case where we have two regions. This is easily generalized for any 
number of regions. The object that would encode the compression for three regions would be Ajyf 3 , for four regions 

would be Aj 01 / 02 ^ 4 , etc. After second level compression over multiple regions, we have 



a kik 2 
^k!k 2 k 3 

V '■ J 



(23) 



There is a third level of physical compression that compresses these multi-region A-matrices to give the Causaloid, A, 
which is defined as 

A=({A}|{A}C{A%,A£*V..}) (24) 

where {A} is determined by the rules of the physical theory (for detailed discussion of how this works see [2]). By 
decompressing the set {A}, we can obtain the A-matrix for any set of regions. This means that the Causaloid gives 
us the ability to perform any calculation that the physical theory allows for. 

D. Well-defined probabilities 

Up to this point we have exclusively dealt with probabilities conditioned on procedures. It is more useful to also 
be able to condition on outcomes. Specifically, we'd like an expression for the following: 

Prob(Y" 2 a2 1 Y" 1 , F" 1 , F 2 2 ) (25) 

Using B ayes' Theorem, this becomes 

Prob(Y 2 \ Yl ,F 1 ,F 2 )_- VToh{Yr ^ F «^ m (26) 

'■2 r 2 

where X 2 2 ~ F 2 2 denotes that the sum is over all possible outcomes corresponding to the measurement F 2 a2 (in R 2 ). 
(For simplicity, we have suppressed the part of the notation denoting the generalized preparation.) In the causaloid 
framework, this becomes 

Prob( Y 2 Q2 \Y?\ Ff 1 > F 2 2 ) = r ° ia2 ' P (27) 

where r Q , 1 ^ 2 = 53s 2 v aip 2 - (The sum being over (3 2 in this notation has the same meaning as the sum being over all 
outcomes consistent with F 2 .) In order for this probability to be considered well-defined, the right hand side cannot 
depend on V — R\ — i?2- Since r aia2 and r Ql ^ 2 are determined exclusively by the physical theory, neither has any 
dependence on V — R\ — i?2- However, p does depend on V — R\ — R 2 . This implies that in order for the probability 



Eq.(27l to be well defined (i.e. not depend on V — R\ — R2), it must vary with p. The dependence on p can be 
removed altogether by requiring that r aia2 be parallel to r Ql ^ 2 . Therefore, the above probability is well defined if 
and only if 

I"qiq 2 II r oi^2 (28) 

With this condition, we get 

Prob(F 2 a2 1 Y," 1 , Ff 1 , F 2 2 ) = J^ai (29) 
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III. r PRODUCT 



Consider two distinct regions; Ra and Rp. By definition 



?p Aap = Tp A ® A Y ap 

Suppose we wanted to take the dot product between two vectors of the above form. Using decompression matrices, 
we can write 



E Ea: 

k A k P \l A lp 
V V 



k A k P | 

l A lp la A\l A a P\l r 



E 4t 



*P A \p r « 



;/ •-ap\u 



where /c^fcp € JIap, Wp € x f2p, and i^ip € Qa x ^p- Notice that we can write 

E A ? Afcp * 



H A l P Va A\i. v a P \ 



(30) 



as 



Similarly, 



Define 



and, similarly, 



E^k A ki 
I'J'p 



E 

l A £Q A 



V P A \,> r a 



CaA \i A [ E tf A ip Fra 



V a P\l> 



Using this, Eq.(30l becomes 



E 

v. en A 



E A$Tr ap L 



ipen f 



-pk A k P / s_ Ai'AfcPr I 



r P en? 



r Ojl«p ' r ^ap — ^ , I ^ ' -^ij ( r «p)^ ( r a P ) v a A \i A v f3 A \y 
k A k P \l A l' 



(31) 



where fc^fcp 6 ^ap and IaI'a *= ^4 x This suggests that the essence of v aAap _jcp Aap is a relationship between 
r aA and mediated by matrices that depend on r ap . Therefore, we can view Eq.(30 1 as kind of product of r aA and 



vp A . Dot products of this form come up frequently enough that we will define this as the T-dot product and denote 
it as 



r aA r(r °p ) v 0A = v aAap ■ Vf3 Aap = (r QA <g> A r ap ) ■ (y Pa <g> A r Qj 
We will make use of this product later in the paper. 



(32) 



IV. CAUSALLY-UNBIASED ENTROPY 



Standard definitions of entropy assume fixed causal structure. Here we develop a causally-unbiased definition of 
entropy in the causaloid formalism. 
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(a) (b) 



FIG. 2: (a) Fixed causal structure (b) Indefinite causal structure 

A. The picture 

Shannon entropy [TT] for a classical state is defined as 

S = -^pilog 2 Pt (33) 

i 

The definition of pi used in this equation requires that the structure of space-time be organized with the following 
features: 

• a region of interest, A 

• an immediate past space-time region, P 

• sufficient data about what happened in P 

• a measurement Fa 

• a set of outcomes, {Y^}, corresponding to Fa 
This allows us to write 

Pi = Prob(Yj|F A ,data F ) (34) 
Removing all time bias from these features of space-time structure, we get 

• a region of interest, A 

• a reference region P 

• an outcome/measurement pair in P. {op} = {(Fp,Yp)} 

• a measurement Fa 

• a set of outcomes, {Y^}, corresponding to Fa 

The reference region can be thought of as a kind of preparation region that is not limited to being in the causal past. 
In fact, the choice of reference region is arbitrary as illustrated in Fig. |2b| 
The definition of pi in a causally-unbiased structure is 

p % = Prob(Y\\Yp,F P ,F A ) (35) 

(Since P is arbitrary, we should technically say l pi with respect to the reference region P\ However, for the sake of 
brevity, we will assume that 'with respect to P' is implied much as 'with respect to the past' is taken as implied in 
the causally-biased situation.) 

Using the above definition of Pi, we define the entropy relative to the reference data (Fp,Yp) as 

S=-J2 ?™h(Y\\Y P , F P , F A ) log 2 (Prob(Yi|Y Pl F P , F A )) (36) 

i 

Notice that this reduces to the causally-biased definition of entropy when P is the past; Fa measures the microstate 
in the classical case or measures in the basis where p is diagonal in the quantum case. 
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B. In the causaloid framework 



Taking the probability to be well-defined, Eq.(29l and Eq.(36l give the following definition of entropy 



(37) 



Of course, this equation requires that v aAap || v^ ACXp . Loosening this condition slightly, we can consider what happens 
when r aACCp is nearly parallel to v^. Aapl using the definition of the probability from Eq.(27l. The entropy associated 
with this is 



s A = -Y( VaAap P ) log 2 ( VaAap ' P ^) 



(38) 



It becomes necessary to shorten the notation for the following work so r aAOlp will be denoted as v, (where the index 
ua is represented by i) and r^.. AOlp will be denoted as u. As with any vector, v, can be decomposed into a component 
parallel to u and a component perpendicular to u (i.e. components in u" and u^, respectively). That is, 



v, 



v- u" 



(39) 



Using the unit vectors as defined, p can be decomposed as 



P=P X U" +PyU 



(40) 



where p~*~ is the component of p that is perpendicular to the plane defined by u and Vj. The probability of interest, 
Pi, then becomes 



Pi 



V; ■ P 
up 

-^ + h 

u 



(41) 



where k — — . Notice that the first term is equivalent to a well-defined probability (Eq. 29 1 . We require the second 

term to be small since the deviation from well-defined should be small. Since we have already required that vf- be 
small, we need only place restrictions on k. 



C. Bounds on k 



For the purposes of this subsection, we will work in the plane defined by u and Vf. Define the angle between u and 
the projection of p into the plane to be 9. Define the length of the projection of p into the plane to be p xy . Using 
basic trigonometry, we get 

P y = PxysinO (42) 

Px = PxyCOsd (43) 

Therefore, k can be written in a form that is dependent on only one variable, as follows: 

k = tan6 (44) 



As 9 tends towards ±5, k tends to infinity. Therefore, to ensure that the second term of (41 1 is small, we require that 
k be finite. Assume it to be a property of the state space for p that there exists some < 9 max- Clearly, |0 mQX | < | 
in order for k to be finite. So 9 is bounded as follows: 

7T 7T 

~ 2 < ~® max — ^ — On"** < 2 

The k corresponding to 9 max will be denoted as k max . Further bounds can be placed on k by the state space of the 
physical theory. For our purposes, it is sufficient that k is finite. 
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D. Q factor 



In light of (41), entropy, as denned in (38 1, becomes 



k-^ 

u u 



log 2 



u 



u, 

,,-L ' 



1 + k- 



io g2 v 4 



(46) 



Since is very small (as is implied by the fact that v\ and u are nearly parallel) and k is finite, we can take a Taylor 
expansion (to leading order) of the first log 2 term. Doing this gives 



-^ + k 
u 



k 

-k 



O v. 



log 2 



log 2 ^ 



Ok 1 ' 



(47) 



Notice that the first term is equivalent to the definition of entropy where u || Vi and that S A reduces to this definition 
when = 0. That is, when u || Vi (or cquivalently, vf- — 0) 



For u/- ^ 0, we will define 



E 



log 2 



(48) 



(49) 



Using k max as defined in the previous section, we can regard k max Q as a kind of correction to the causally-biased 
entropy. Then, to leading order 



S krf. 



< S < S + k max Q 



(50) 



E. Understanding Q 



Q is an entirely new quantity with no direct classical analogue so understanding its physical interpretation is a 
non-trivial matter. If we consider entropy as a measure of uncertainty, then 5* is the measure of our uncertainty that 
the measurement Fa in region A will yield the specific outcome Y\, given the data we have from the reference region 
P. Since our reference region P is arbitrary, one way to view Q is that it measures how completely the region P 
"prepares" region A. In this sense, preparation influences our uncertainty. In a definite causal structure, an immediate 
past region would completely prepare our region of interest and Q would be zero. However, in the causally-indefinite 
picture, we cannot require a priori if the reference region that we have chosen will completely prepare our region of 
interest. If there are no influences on our uncertainty from outside region P, then the probability will be well-defined 
and Q will be zero. But if there are influences on our uncertainty from outside region P, then the magnitude of Q 
will reflect that. 



F. Using the Q r product 

For the sake of completeness the u's and v,'s must be translated into r^ AQp 's and r QAQp 's. Notice that 

v[ = Vi ■ u 

u u u 

v£_ = vf -v\ = / v; ■ vj (v; ■ u) 2 
u V u 2 y u • u (u • u) 2 



(51) 
(52) 
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AOip 


for 


u gives 










J 




v aAap 


r 


~*Adp 






u 






■ r 


— ACtp 






v± 




1 r a A a P 


^acAOtp 


{ r a A ap 




u 




V v -~- A ap 




( r ^ A ap 


■ r ) 2 

1 ^AOlp) 



Using the T-dot product the above equations simplify to 

l OLA K ~> L ^A 



u 



u 



r_ A r ( r °p) r. 



\ 



r aA Q r ( r °p) r c 
r_ 4 r ( r °p) r„ 



r(r Qp ) 



(53) 
(54) 

(55) 
(56) 



This allows us to completely specify the entropy of Ra relative to a preparation Rp in the causaloid framework. It 
is straightforward to generalize this to define the joint entropy of Ra and Rb with reference to a "preparation" Rp. 
Simply redefine u and vy as 



where 



and 



u 



a A a B ap 1 a A 



(57) 
(58) 



Pa 



I3 B 



Using the same procedure as for one region, we get 



u 



n _ 
u 



r (7) r ( r Q P ) r 



r(r Qf> ) r 



r « r ( r °p)r 

1 a A a B w 1 aAOiB 

\ r o r ( r »p) r 

\ ' *A ' ' B u '■'-A'-B 



r o r ( r °p) r 



(59) 
(60) 



In this manner, we can define causally-unbiased entropy in the causaloid framework for any number of regions. 



V. CONCLUSIONS 



In a definite causal structure, the only thing required for a definition of entropy that is not in an indefinite causal 
structure is an immediate past region. Since there is no reason in an indefinite causal structure to choose any reference 
region over any other, we simply choose an arbitrary region. This ensures that we do not hold on to any pre-conceived 
notions of space-time and its connection to causality. The definition of the causally-unbiased entropy resulted in a 
correction to the causally-biased definition of entropy. In a sense, the Q factor gives us an emergent idea of causality. 
It is a measure of the extent to which our region of interest is causally connected to our reference (or "preparation") 
region. If it is zero, the traditional ideas of causality are recovered. The next step would be determining how 
the Q factor could potentially be physically observed. To do so may require us to know more of the theoretical 
and mathematical properties of Q. Which mathematical properties of Shannon entropy hold for causally-unbiased 
entropy? What is the status of the Second Law of Thermodynamics in an indefinite causal structure? To go about 
answering this, we could consider how S A "evolves" along tubes through indefinite space-times. These questions will 
be the subjects of continuing work in the near future. 
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